The present invention relates generally to biometric techniques for sensing stress and emotion in human subjects and, more particularly, to non-invasive, optical systems and methods for sensing stress and emotion. The underlying principle on which the present invention is based is that internal emotional and physiological states of a human subject are correlated with transient and vibratory surface motions of the skin, and that these motions provide insight into the subject's physiological and psychological state. Moreover, this insight can enable a reliable assessment of the affect and credibility of the subject through the application of algorithms based on established databases.
More specifically, the relationships between emotions and a subject's facial expressions and movements have been well documented by Paul Ekman and others. The Facial Action Coding System (FACS), by Paul Ekman, Wallace V. Friesen and Joseph C. Hager, originally published in 1978 by Paul Ekman, University of California at San Francisco, San Francisco, Calif., includes an ACS Manual and other materials defining the emotion-expression relationships in detail. Dr. Ekman has authored or edited numerous other published papers and books in the same subject matter. See, for example, “What The Face Reveals: Basic and Applied Studies of Spontaneous Expression Using The Facial Action Coding System (FACS),” Ekman, P. & Rosenberg, E. L. (editors), Oxford University Press, New York, 1997.
The best known prior art technique for making such measurements of facial features and movements was based on the use of laser Doppler vibrometry (LDV) and relatively primitive electro-mechanical sensors. LDV is a technique for detecting surface vibration without physical contact and is widely used in engineering and manufacturing applications.
As useful as LDV has proven to be for making measurements of facial movements, the method has significant drawbacks. A laser beam must be pointed at a specific test point on the subject's skin that is believed to be indicative of an emotional state. The vibrations or transient motions induce frequency modulation on light that is reflected back to the instrument, and well-known heterodyne laser detection methods are used to recover the mechanical signal for display and processing. The beam must be maintained on the test point to make the measurement, and this requirement presents some practical problems, in general requiring that the subject's movements be constrained to some degree during the test.
A more serious limitation is that only a single point can be examined at a time with conventional LDV hardware. While it is known that monitoring many points on the subject's skin simultaneously provides significant information concerning the subject's affect, a separate LDV instrument would be required to monitor the response at each point to capture all of the spontaneous facial movements needed to detect a particular emotional condition. The pioneering work of Ekman and Friesen on micro-expressions showed that many facial muscle groups are involved in the involuntary transient expression of emotion. For a brief time of about 300 ms (milliseconds) or so, the face of a subject responds involuntarily to emotion-evoking stimuli such as questions, comments, or scenes, and these fleeting expressions can be detected and interpreted. The Facial Action Coding System developed by Ekman et al. maps specific emotions to coordinated motions in more that twenty facial muscle groups. Hence, being unable to monitor all or most of these muscle groups at the same time is a serious limitation for LDV instruments.
Lastly, signal processing for the received signal in an LDV instrument is somewhat problematic, requiring algorithms to remove subject motion artifacts. The LDV is useful for making high fidelity measurement of skin surface movements for cardiology and other applications, but it would be highly desirable if the motions of an entire face could be sensed at an instant, so that correlated responses among muscle groups could be detected and processed. The present invention is directed to this end.